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ABSTRACT 

Kine independent groups of students totaling 863 
subjects made a metric judgaent of 13 concepts (TV programs) in a 
study to assess the'iselecticn of the best criterion pair in a pe^^ric 
multidiffi^nsios^l scaling (MDS)- tast— Hsin-g-tie GALILEO metric MDS 
prograi, judgments were scaled and plotted in t hrpe-dimensional 
space. The three hypotheses were confirmed: (1) the structure 
produced by a criterion pair involving the extremes of the concept 
domain. was statistically identical to the structure produced by the 
same scale with no concept anchors; (2) as the distance between the 
criterion pair was increased,- the. judgments of distances • among ^ 
concepts increased b»t the pattern remained +ie same; (3) a criterion 
pair that was close together produced a larger < structure tha*n an 
extreme pair, yet concept interrelationships remained stable. These 
results are discuss.ed in teros of the subject^s ability to use metric 
HDS and of the reliability and validity of the technique, 
(Author/AA) 
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INTRODUCTION 



As Schramm '(1973) recently noted: ^ I 

The main effect communication has on ys is on the picture^ in our he^QS, 
• "^r^^gnitive maps of environment, our- images -ot ourselves, the beli-effs 
and vat'ti^ -we have accepted'and are* prepared"' to defend, the evaluatidns 
Ve have ma^i&^f our relationships to indiyidiikls and* groups. (p. 194). 




r 

' Multi(^imensional scaling (MDS) provides the too.1 for the creation and explor^ion 
of these cognitive m^s and their relate^ ^communication/cognitive processes. 
For mass communication research, the power_ of MI^S rests with its ability to, » ' 
provide quantitative . multidimensional models of cognitive processes which can ' • 
be applied to ^a variety of mass rae'dia effects area's^ In essence, examining an 
MDS plot ^representing relevant audience conceptions is like looking' at the 

• . ' ^ . ' ^ ■ . ^ ' . r 

a^verage mind of the listener/reader/vieWer : Studying, iij a -precise quantitative 
sense,, the interrelationships of .these conceptions .appears to hold exceptional 
potential for explanation, prediction, and understanding of communication related 
phenomena. • ' * 



Approaches to MDS . ; ' . ' * ' ^ • 

Although a variety of scaling techniques ^are available ^or MDS, Shepard (1972a) 



^notes that: 



The unifying purpose that these techniques share, despite their diversity, . 
is the double one (a) of somehow, getting hold of yhatever pattern or structure 
may otherwise lie hidden in a matrix of. emtpirical data and (b) of representing 
that structure in a form that is much more accessible to the human eye-- 
nar^ely, as a geometticaJL model or pictute (p. 1). ^ 
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The two major .approaches to MDS are referred to as the ''classical" or "metric" 
approach and the "nonmetric" approach. These terms characterize the level of 
measurement of the data used in.^the scaling, procedures. In the metric case interval 
or ratio data are required vhereas in the nonmetric casfe ordinal- data are sufficient. 
Building on the mathematical groundwork of Yoithg and Householder - (1938) and 
Richardson (1938) , .Torgersqn (1951; 1952) is credited- with perfecting the metric 
formulation of MDS and his book published in 1958 has become the classic Statement . 
The first computer routines for nonmetric MDS of ordinal data were introduced 
by Shepard .(1962a; b) and later refined by:Kruskal (1964a; b). ' 

' ^ • . • f - 

Nonmetric vs. Metric: Advantages & Dis^advantages 

Nonmetric. In noninetyic MDS ordinal" Judgments pf the similarity of a set of 
elements are obtained (this 1.3 tlie most^ common form of data collection although 
Others are possible, . sfe^ Shepard, i972b for other , types) . These elements are then 
modeled by systematically changing t^t€ir structural relat;ionships (an iterative 
process) until they "fit"- (show a linear or monotonic relationship with) the original 
ordinal data. The resixlfring -p^^sitxp^'^'ot the elements are 'thus transformed from . 

ordinal information £o intgjrVal and^ thru an additive constant, to ratio representations. 

j J ' 

[There are a variety^ of, nonmetrtp MDS routines available. , Of those using aggregate 

data (most relevant for tq^ss media*) Shepard (1972a, p. 8) nbtcs: 

1 ...when they are applied to the same set of data, the various methods of 

this general type. . .usually yield virtually Indistinguishable results. 

after' all, monotonlclty is a rather well-defined notion and any reasonable 

. -way of evaluating It could be expected In prBctlcc to lead to similar results 

♦ « 

J^onmetric advantages (Shepard/1972a) : 



1. Metric representations (models) -are derived from ordinal information. 
2: The Judgement task for the respondent* is easier than In the metric case. 
3. The method can be applied to situations where collection of metric 
information may be questionable. , - * 



Nonmetric * disadvantages i L ' . » - . - 

1. The model is produced thru Iterative (repeated) adjustment of the structure 
jfor a best fit \>ith the original data. Since these data are ordinal, the 
model is still, constrained to fit only the orderings ^f the judgements* 

' (Woelfel,.1975). ' . ' \ . * 

2. ^ There should be at least two to three times^as many elements tor be structured 

as there are dimensions if an optimum fit with th^ original data is to be 
ob.tained (see Sherman, 1972; Spe'nce, 1972; Young, 1970). Thus, to decide 
how jzany elements to use, some prior knowledge of the dimensionality of 
the' structure would be beneficial (Danes and Woelfel, 1975). 

3. ''"Stress" (lack of fit with original data) increases as the number of elements 

being scaled increase's (Young, 1970); yet, the metric retrieval becomes 
more precise (Danes and Woelfel, 1975). . ^ 

4. The iterative and algorithmic transformations require more time and expense 
than do the^ metric routines. * ' 



Metric MPS . Metric MDS procedures elicit Interval or ratio judgments of the 
degree of similarity of a set of lelements. Since, in the aggregate ca,se, the distances* 



between these elements are obtained as mean distances, they can be scaled directly ^ 
with no need to go thru the iterative process of nonmetric MDS in order to obtain the 

K 

final structure. In the interval data case, the addilrlve constant transformation would 

be needed to obtain ratio data for the final structuring. If ratio judgments , are 

[ 

initially elicited, no dita 'transformations are irequired. Thus, the preferred case 
would^Jbe ratio *judgment^ of the .similarity in terms of distance. The following 
advantages- anH disadvaiita ges are b"a^ed on this paired ceaparifion tatio ilttric approach. 
'XbjLs-^pproach begins by giving^he subject a criterion or standard which can be used^ 



to make all* other cojnpari sons. For example, the subject is./tolrd that e,l«fiients A. and B 
' - • 1 * r/ .? ^ •* * - 

are X units apart ..vj&iow^ng this, they judge A from C, C from B, etc. for all pa^rs^^ 



(see tjie i^ethods section 



point.. fsNrstab listed 



of this paper for complet^^ ijvstruc^HC^ns). >^e ab^olu|:« zero 



by ijnstructing the subject tb regard compliBfee identity betjjjfien,; 
any" two elements (zeTo dissimilarity) as zero distance apatt.. . ^- , , '■ - 



Metric advantages (Woelfel, 1974): 

1. No ififormation is lost in the creation of the structural whole from the , 
original ratio data. 

2. Since no iterations or transformations are needed^ a definite cost-efficiency 
benefit is gained. 

3. Comparison of structures across samples and/or over time is facilitated 
if a consistent ratio scaled standard of judgment is presented each time 
judgments are requested, . 

4. Relative to the aggregated ratio judgments, as opposed to interval or ordinal 
judgments: 

a. The scale is unbounded at the high end, continuous, and po,ssesses an^ 
absolute zero point. 

b. As an aggregate or sample mean, thdse distances between elements are 
the best estimates of the true population conceptions. » 

c. Error in mean estimates can be expected to be random, normally distributed 
. about the mean, and inversely proportional to sample size.^ ^ ' 

Metric disadvantages : * 

1. Ratio scaled metric judgments are more difficult to make than ordinal judgment 

2. If judgments are more difficult, reliabifity should ^suffer. To overcome this 
^larger sample sizes may be needed. 

3. In the ratio technique, criteria for the choice of a standard or criterion 
pair to which all other judgments are relative have yet to be empirically 
determined. Also, information about subject abilities to apply a given 
criterion standard in mapping out their cognitions is minimal. 

On face value it would appear that the advantages of the metric approach would 

make it a clear favorite. However, the disadvantages should not be taken lightly. 

In the aggregate metric case disadvantages one and two above can be overcome by 

utilization o^ a Sufficient sample size. , The third disadvantage above is central to 

the-^* application of the technique. The present study deals' with this problem area. 

THE PRESENT STUDY 

-✓*-.» ^ 

^ ^ ' ' ■ . 

The sel6ct*i6h'' of a suitat'le standa^rd or criterion pair on which all other 

• :. " • .... . ^ ^ 

judgments are based^ in the ratio metric approac^h appears crucial to the technique* 

since this cxitition tecome^ the ruler used hy the respondent to measure their , 

' ^ '\, ' •* ' ^ ^ ' 

cognitions for the researcher. The criteria available for the^ selection of this 



standard are minimal, Woelfel (1974a, p,*16) suggests the following: 

t. ^ f " \ ' 

^ First, the standard* should be relatively strfble. Changes in the _ 
standard over time. can coqfound time series measurements and pre- 
vent meaningful comparisons of measuifements made ^t different times. 
Secondly, the standard should be the same for a<ll observer's rc^ardr.^ 
less of reference point, i.^., two independent observers must both 
agree onJhe length, for example, of a meter or a kilomelter. Less 
imppftant, but nonetheless worthy of consideration, good practice, y 
for minimum error suggests using a ^standard approximately midway J 
bet^^een the largest and smallest measurement likely to be encountered, 
(measurement of astronomical distances in miles, for example, is 
cumbersome, as would 'be measurement of terrestrial distances in 
fractions of light-years). * 

The present study builds upon the la&t oi the above suggest ion^in assessing the 
respondents ability to u^e standards of differing unit lengths (10, 25, 50,' 100) 
involving* pairs from the extremes of the^ concept do^iain or pairs closer together 
in that domain. Jit the same time, the findings of* an earlier study by Gordon-^nd 
De Leo (1975) are replicated. * 

The Gordon and De Leo study determ ined that providing a 10 unit criterion 
pdiiT composted of the extreme's from a homogeneous .concept domain produced a structure 
which was statistically identical to using the concepts "red. & white" as a criterion 
(10 units apart) and/or ualng only a teti unit scale base with no anchor . concepts 
(with*the option to make judgments larger than 10 units). The interpretation of 
those results suggested' that for the red-white* and no concept treatments, the only 
u^^able information for the subject was the 1*0 unit scale base since color was 
irrelevant to the concepts being judged. Thus, judgments were made' with that sd^le 
base in mind and identical structures resulted. With the extreme concepts as the 
criterion, although Ithese concepts were meaningful, the fact that thtey, Represented the 
extremes produced the same result by essentially limiting, the judements to that base. 

The present study was designed to replicate the*. finding that the criterion pair 
consisting of the extremes from the concept domaJ.n will produce a structure identical 
to providing the same scale })as^^ith nc concept anchors. As well, the study ex^nds 
this to varylthe distance between th/^concepts to further explore the subject's 
ability tii^Jjli'^)! ^the criterion given. . ' * 



Overall, it was expected that th6 same relationship as evidenced in the earlier 

study by Gordon and De Leo (1-975) would hold true«» Thus the prediction was: 

H^: Given the same unit distance, the no criterion pair and tfte.' extreme concepts' 
criterion pair will produce statistically identical structures, . 

Also, given the evidence concerning the reliability of metric MDS with aggregate 

data (see Barnett, ^1^2r^illham and Woelfel, 1975), it was predicted that: 

* " .- ' 

As the distance between the criterion pair *is increase.d, the resulting 
judgments of distances among concepts will increase- but the pattern of 
Concept interrelationships will remain the same, * 

In the Gordon 'and De Leo study a fourth criterion pair treatment was employed 
» 

using two concepts somewhat closer together in the concept domain; The results of that 
comparison suggested that a:c?:iterion pair using concepts that are close together 
will produce an expanded structure sinc^ mopt of the judgments must be made outside 

♦ ar 

(larger than) that distance baSe, Thus,*. the prediction was: 

H3: A criterion pair using concepts close together in the concept domain will 
produce an expanded structure (as compared to the extremes), yet the 
intertelationships ^ the concepts will remain the same* 



METHODOLOGY - 

Concepts . , , 

As a methods^ study, the parti-cular concepts used' for the scaling 

comparisons were of secondary interest. The concepts selected were types 

- . ' ' . \ ^ - ? 

of television programs and -particular titles of ^shows. related- to .each . ^ 

program type. These' choices were made, on the reasoning that most; subjects 

would be able to judge these^ concepts and, as* such, maximum judgements would 

be obtained. Also, the clusterings of program^ t itles with progr^am .types 

could be examined for internal content validity of the resulting ^structures . 

Six general pr'ogram types and six re^^ted^^s^ plus the concept "me", (self) 

resulted in a total of 13 concepts requiring 78 paired judgements. The 

particular TV shows were selected bn the basis of having high ratings 

in a recent ratirrgs period'^ (Broadcasting, p. 19). A listing of^th6 concepts 

/ . ^ ^ 

* fpllows : , * . ' ► * 

1. Children's Comedy ' 7. Fat Albert V 

. 2. 'Adult Situation Comedy , 8. All In the Fsgdly , ^ 

,31 Soap Opera /"^^^ General Hospital 



13. Me 




4. Family Drama* * IQ. The Waltons 

5. Medical Drama ll\ Medical Center ^ ^ 
^6. .Crime Drama "^V* 12- The Streets ^L%an Francisco 



Treatment Conditions " • - ' " \ . 

To explore the effects of criterion pair variation and to partially 

replicatp^ the findings of oup earlier study (Gordon and De Leo, 1975)^ 

three' major/ treatment variations were employed. First , the extremes of 

the concigpt domain were* used and thes^e. were specifffed^gs dif fering dist;^nces 

apart for different treatment groups. .Secopd, two concepts closer together 

•.. . . ' ♦ , , # 

in the domalh were used, vapying their specified distance; Thfrd, no concepts 

' • * ' * 

were used but instead subjects were simply told, "As you judge the distances. 



keep a ten point scale in mind some shows may be less than ten units apart 
and others may be mor,e." The nine specific treatment conditions (independent 
groups) were as' follows: • ^ 

1. None (no anchor concepts, only a 10 point scale base) 

2. Children's Comedy - Crimp Drama = 10 (CciO) 
.3. ^ M ^ ' „ . " ^ = 25 (CC25) ' . 

4. • , ' " * " " ' = 50 (CC50) * - . 

5'. " " ♦ :j= 100 (CClOO) 

\ 6. Family Drama.— Medical Drama = 10 (FMlO) 

7. „ ^ 25 .(FM25) / , ^ ' ' 

8. . . " " V • =50 (FM50) 
t 9. " " ' / ] 100 (FMlOO) 

Subjects 

A tot^l of 863 students were the subjects in this experiment. The number, 
'in each treatment ranged from 92-112^ The, large numbed of subjects was deemed- 
neces^^,ry so that the comparisons would be based'.on stable s,tructures. Tab^,^ 1 
provides the n-s for each treatment. , The students .were randomly* assigned 
(by classes) to treatments. The departments sampled included Anthropology, 
Ed'ucatipn, Journalism, Psychology, Radio-TV-Film, Sociology, Speech, and Theater. 

Relative to 'the distribution of^subjects to treatments (see Appendix A), 
Chi Square analyses on the ngiedia related variables 6f '(1) "average hours of TV/day,, 
(2) average hours TV/week, and (3) preferred .tjrpes of TV progi^am^^the distributions 
of subjects in all trea^tment groups were equal. TKis is true also for the demographi 
variables of age, sex, and fami^ income. On year in school, the random assignment 
of classes to tteatments resulted in treatment CQSO having upper class students 
overrepreQented--all other treatments were equal. .On face, CC25 ^nd CCIO had fewer 
Blacke than did tjie other treatments.;, . 



Procedures ^ ' \ , 

* '* ^ 

The data were, collected December 1-12, 1975» S.ubjects in classVooms. 

were given one of the nine treatment variations and^ the following instructions 



were read with them:, / ' . / - • 

This form asks you to t^ll us how different (or in other words, "» . 
how "far apart") TV shows are from e.ach other. liiffetence^ .between " • \ ^ 
shows' can be measured in units ^ so that the more different two shows ^ \ 
are, the more units apart they are. To hel^ you know how big a unit , / 
is,^ and . ar>e " units apart. 

You are supposed to tell us how many *unils apart thb shows on the^next ' 
/ few pages are. Ranenber, the more different the shows are from each 

other, the larger the number of units apart they ar4. SOme shows may • 
be more than . ^ y units apart and^ a^ome fnay be les^;.\ > . 

Npte that: • *l ' '» - 

' — "jtie" on the questionnaire meansyourself Juagem^nts involving '"me 
should indicate how close y/Ou feel to that TV\ show or type of 
' program. ^ , , 

-r-^Zero can be used as a distance; if ybu see two things^ as identical 

they would be zero distance apart* 
— If you are not familiar f^th a TV show or type of program, leave 
that pair blank. 

Please work 'quickly. Judge the^ shows as pairs rather than trying to 
relate each judgement to all oxhers. 



Blanks in the above instructions wefe filled by the criterion pair used in 

a particular treatment. For the -no criterion pair treatment the last sentence 

in* the f irst .^^^.graph of the instructions read, "As you judge the distanqes, 

keep a tep point scale in mind^ — some concepts may be less than ten units 

» * • * 

apart and others may be more," ' 

On the average, the items were completed in 15-20 minutes. Most; of, the 
Subjects were able to judge the 78 pairs with the average number .of suBject 
judgements ranging from 83.87 to 106.74. 



. " RESULTS ,• - 

Individual Treatments ^ 

Using ver$ion 3.0 op the GALILEO metric MDS pfpgram, treatments were 
■first processed ihdividually . Appendices - prS^de .the summary 
statistics for each treatment and the resulting normal factor solution 
which cfefined the concept locations a$ coordinates* in three-dimensional 
space. Table 1 provides the percent of real distance accounted for by the three 
factors, the imaginary distance of the total solution, the trace values, 

^nd the average distance judgment for each treatment. 

i * . f 

\\ ,To avoid the effect of extreme values on the means, maximum values 

\ 

were set for each treatment using the maxlm^ value option of the GALILEO 
program. These valutas were determine^ by successive runs in which the 
extreme values were gradual 1>^ reduced while dbserving the minimum-maximum 
descriptive Statistics as related to (a) the criterion pair given; (b) the 
means; and (c\'the standard deviations* , The maximum values finally used 



(cVthe stand 
>d in Table 1 



are provided in Table 1 alortg with the number of judgements that value excluded, 
and the* average number of observations remaining per cell. 

Given the scaled concepts, plo-ts of- the first three orthogonal factors 
were obtained^ using the plot option of the program. These plots are presented 
in Figures 1 The high , decree of similarity of inter-conetf^ locations 

across the treatments is visually evident in these figures. Keeping in mind 
the mean distances^^ these plots show that t¥fe concept locations are highly 
similar for each -^treatment , "while the actual mean distances between concepts 
differs considerably^ across treatments • 

Comparison >y£ . Treatments ^ / " / 

frjelations. To statistically, verify the consistency of concept . locations 



ac/oss treatment^, the mean^lstances fo^ the 78 pairs in each treatment were 
entered as scores' into-^standard' Pearson product moment cor relation. Thus, 



'li- 



the patt^rt^-eyidlnt in the inter-^'concept distances could be compared across 




treatmet\t»s* TaM«^2^-pa:esent5 the results of these intercorrelations . 
In all cases the cori>elatr5n5..Z§re.^tremely high-, i^anging from ^933 to .988. 
. ^ . -Of course^^ these exceptionally high correlations arid the high degree of 

linearit-y evident'^^om -the plots suggest high predictability. 'For example, 
^ a simple two .^riabl^ linear regression of -the criterion pair distances as 
_ independent varf^les (within a CC or FM treatment set) on (1) the average distance 
judgment, (2) the tr4ce values for each treatment, (3) the overall sum of squares 
for all matrix values in each treatment and (4) the average distance moved for the 
concepts in each treatment, as dependent variables produces r (variance e^lained) 
values ranging from .966 to ,999. 

Figure 3 presents the comparison of /spaces plot for only the CClO and NONE 
^ treatmept^, ftoth having a 10 unit base.y It is (3fcvious from this plot that hypothesis 
one (pTfedictrng statistically identi^^al structures for CClO and NONEJ is confirmed. 
Thts result replicates the findings of our earlier study^ 

Comparison Mr spaces . »Next, the treatments wfere compared using the 
Comparison of Sp^Bs^ option of the GALILEO program. This option uses a 
least squares rotation of the axes of the individual treatments to one 
treatment specified as the mainspace. For this^plot, the treatment 

organized by trace to keep the plot as neat ^s possibly, given the c\mpld^ity 

i - 

of comparing nine data sets. Thus, FMlOO was defined as the mains] 
(tifis decision wSs made, rather than using CClO as the mainsp\ce» b/cause the 
concept numbers are placed with the first plot anci the CClO plot wou-ld have 
placed 13 numbers into a very small spa<|e). The trace ordering was: CClO", 
None, FMIO, CC25, FM25, CC50, FM50, 'CClSfCr, FWlOO. Figure 2 shows ^t he resultijig 
plot and confirms the intercorrelation interpretation that the relative concept 
locj^tions are highly similar for each treatment -but the space expands as the 
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1 ^ 



criterion pair distance increases. , As predicted, for the same given distance, 

the criterion -pair which is closer together produces a larger space than does 

' ' 2 
the extretSwjf pair . ^ - 

FigiiFe^3 presents the comparison o^f »pac€? plot for only the CCIO and None , 

treatments, both having a 10 unit base. It is obvious from this plot that 

hypothesis one (predicting statistically identical s^x;{Jctures for CClt) and None) 

is confirmed. Thi# result r'^plicat>«« th« Endings our eatsUer study. 

Distatice Judgments Relative to Criterion Distance 



The corresponde"nce of the average (observed distances in a resulf^lng structure^-- , 
relative to the criterion pair used to elidit those structures should p^it^ide useful 



To examine this relatipnstrtp" 



information for^-tKese^ection of the critel 
the concepts involved- in the pair were also included vithin the concept listV 



The actual average judgment of the, concepts involved in the .criterion ^ir, for 



each treatment are given/fTelow: 

Ave. -Judgment^^ 



Treatment 

NONE 
CCIO 
CC25 
CC50 
.CClOO 

FM 10 ■ 
FM 25 
FM 50 
FM 100 



CC concepts 

t.02 
9.54 
23.09 
48,83 
97.16 

6.47 
17.15 
27.60- 
50.97 




3.53 
7.85 
22.40 
49.03 



. • It is clear from the above that the .pair involving the extren»es given as* 10 units . 

' ■ . — , 

apart (CCIO) produced the structure (o^ that pair) which moat closely corrffffpondB-ta — .-_ 
the criterion distance given. The guestion of vbether tfeis sing-le pair is representative 
since it was given to the subject is answered by the r^cognitl^ that it was given 



to all treatments yet there are gross differences across tre^jU^tsr and, at the 

same time, the j(^ntercorrelations of the structures as wholes are extremely high. 

Since th,e FM ^concepts were quite close together in the space, .it would be expected 
• • • 

that any movement in that direction by choosing other than the extremes would 
reciuce th6 distance averages overall, thus, it can be concluded that if the absolute 
distances in the structure are pf interest, a criterion pair involving the extremes 
would be the best choice. 

Variability of Judgments 

The variability of the concept pair judgments across treatments was compared 
thru the coefficient of variation. This coefficient, which is the ratio of the 
standard deviation over .the mean was used rather than, a direct comparison of 
variances because of the differences in scale b^ses across treatments. Thus, 
'rat^.os were calculated for each concept pair and these were averaged across the 
78 pairs for each treatment. Table 3 presents the mean and .standard deviation 



for 6ach treatment, the one-'^ay ANOVA>^^mparing all nine treatments, and the selected 

comparis^ons (Scheffe^s procedure). 

y Following from the significant overall ANOVA, the selected comparisons 

demonstrated that the only significant differences (p<.05) were between CCIO vs. 

FMIO and GCIO vs. FMIOO. . Thus, varia'bility of judgments relative to the mean of a 

given scale base revealed' that CCIO produced the smallest variance ratio with FMIO 

and FMIOO producing the largest. In general, this ratio was larger for all FM 

* 

* treatments as 'compared to' the CC treatments, though not significantly so in most cases. 

"Skevn'ess of Judgments » • * ' 

The skewness of the distribytions' of judgments Vithin each treatment were^ 
QompWed by averaging the skewness values of each concept pair^vtCfiln each treatment. 
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Thus, since there were 78 concept pairs in each treatment, 78 skewness Values 
were averaged. The mean skewness values were then compared hy (1) comparison 
to a normal distribution, (2) a one-way ANOVA across the nine treatment groups and 
(3) selected comparisons using the ScheffI procedure. The m^an skewness values and 
related standard deviatipns appear in Table 3. ' - 

The skewness values demonstrate first that all treatments^ on the average,- 
were sjcewed signLf icanCly, to the right as compared to a normal curve*. This skewness 
is due primarily .to the effect of the larger distance judgments In extending the 
right tail of the distifibution. Thus, as the criterion pair distance j^reases, 
the extent oS the skewness to the right also increases. Ove^r^lirr~CC10 wa€ the least 
skewed though ^till significant' at th^ .05 level. The skewdess of all other treatments 
was at, or beyond, the -.01 level. . ( ' 

The AIJOVA comparing the nine treatment groups on skewness was -significant (p<.001). 
The selected comparisons demonstrated that all other treatments .differed significantly 
from CCIO (p<.05) with the single exception of ,CC50. In terms of general consistency 
the criterion pairs involving, the extremes (CC) were less deviant; thaa were the closer 
pairs (FM). Here, CC25, CC50, and CClOO did not differ significantly from each other. 

. Kurtosis of Judgments * ' 

The degree of kurtosis of the dlstribtrtions of judgments, on the concept pairs 
were compared thru analyses similar to those -for skewness. The kurtosis values for each 
concept jjair were averaged for the 78 pairs in each treatment. The mean kurtosis 
values were then examined by.(I) COTiparison to a normal dlstributiion, (2) a orie--way 
ANOVA acrpss the nine treatment groups and (3) selected comparisons (Scheffe's). 
See Table 3. 
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Validity checks . Two elements were built into the design of the instrumentation 
to allow for validity c^iecks on the structuring of the concepts. First, as a type ot 

« 

content validity check the particular television program titles were expected to 
cluster in the space near their general program type. It is obvious from the plots 
that this is the case. Second', as a form of construct validity check,' an open ended 
item was included asking the subject to indicate', in general, what types of TV programs, 
they prefer. A summary of responses to this question and the significance test 
for differences between ;:reatments appears in Appendix A. The significance test 
shows that the treatment groups are, equal in terms of program preferences of the 
subjects in the study. Also, of the program types used, the preferred types were adult 
situation comedy and crime drama. As can be seen in the plots, the^"me" or self- 
c&ncept is located between and nearest these two types. The fact that the self-concept 
is somewhat distant from all programs, in general, is evidenced by the fact that a 
substantial number of subjects indicated preferences for types of shows not used in 
this study (see Appendix A). Thus, although the validity checks are minimal, the , 
structures are highly inteirpretable in terms of both the content and construct checks.. 




CONCLUSIONS AND DISCUSSION 



Hypotheses Cohclusions 

T^e^esults\^ this study confirmed all of the initial hypotheses: 

First, the rto concept criterion treatment produced a structure vhich, relative 
to mean distances, was essentially identical to the structure produced 
by the extreme concepts criterion pair (see Figure 3). ' 



.Second, as the distance between the criterion pair was increased from 10 to 25, 
50, and 100 units, the resulting structural space was increased--yet the 
interrelationships of the concepts to each other remained the same 
(see Figure 2). 

4 

Third, Given the same distance between criterion pairs, the pair close together 
in the concept domain will produce a larger overall structural space than 
will the extreme concept pair. This results from the fact that more 
judgments must "be outside or larger than that unit base. 




Subanalyses Conclusion^ 

Further analyses Qomparping the judgments r'esulting from the extreme criterion 
pairs (as opposed* to the closer pairs) showed (A) more consistently high intercorrelation 
among all structures, (S) less variability of /judgments, (C) less ,skewness, (D) less^ 
Jcurtosis, (E) more interpretable absolute distances relative to the criterion pair 
'distance given^ 

Within the extreme pair set the least deviant judgment distributions resulted 
from theCClO and CC5p conditions. CCIO produced the lowest variability of all pairs 
^{though not significantly different from any other CC pair), the least skewness, and 
/the smallest absolute difference between the criterion distance and the resulting 
observed distance on that pair. CC50 however was just as low, in variability apd 
skewuess, and produced the only distribution not significantly different from normal 

on kurtosis. The CCIO distribution was the only treatment significantly flatter 

^ f, 

(platykurtosis) than normal, . ' • 

"\ ■/ - • 



The 25 unit distance 'as . an odd nuJnber, .appeared to ]be more difficult to use.,^ 

# '*.*'* 
This distance, for the* eSitteme pairs , produced the moat excluded scores on the maximum 

J? 

value limits-- and the secXfhd highest variability, skewness, and^lcurtosis values.,- 
The lOp unit base^ produced' i'He most skewness knd- kuetosis^f the extreme ;pairs, 
I The frequency distribution of judgments for CGlOO revealed th^. dlmost all judgments . 
werfe made in*' ICf unit jumps^wifh some 5. unit cHbices between. This continued up to 
the 130 «nit/ dl^tatfce and irom there <t^e judgments were in 50*^s and lOO's. * Thus, : 
with 100 units the. [full a(Lale is farely used and the larger jumps at the e^^tremes * 
contribucg *to the distortion " of the distribution. . - 



Discussion 



The results of this study ^ relative to -the ability of metricMDS" to reproduce 
a consistent structure as operationalized thru the' GAl^ILEO program, and , the subject's ,^ 
ability" to adapt to a -given criterion are very impress tye. The o.dds tliat nine V' . , ■ 
independent. groups of people, each using different criteria to judge 78 different pairs; 
of concepts, would produce statistically identical structures must be very Tow. Thus, 

" . < • . ) . ' ^ ■■ -V 

these results shed light on both the subject's ability to adapt to differing measurement 

criteria and the method's ability to precisely reflect those judgments. Error on . 

either side would have decreased the liljelihood' that similar structures would be 
evident across treatments. t '• . 

It is clear that for the choice of a criterion pair for use' with a homogeneous 

set of concepts, the extremes from the" concept domain are prefei;abl,e over a pair 

t • • • ' 

> - • .1 

close together in the set. Given the evidence for interpretation of the absolute * 

distances produced from the judgments, thi;5 also appears , preferable over the suggestion 

to choos^i^ standard^ "...approximately midWay between the largest and Smallest ) 



me 



a^urement likely to. be encountered, (Woelfel, 1974,.,p. 16)» . 



The best choice* for a unit distance between concepts in the criterion pair, 
of thos^ tested, appears to.fce,elther 10 or 50.. Ocx^ the basis of the distributions, 
50'would be preferred. However, we should note that in th^ methods section It WUs 
pointed out that the random assignment of • claases, to treatments resulted in an 
6vefr representation of upper class -students in the CC5&^ treatment . ^ Thus, the ' - 
greater consistency of CC50 in terms &£ kurtosis may reflect the greater degre^ of 
homogeniety of that group and their more advanced' cognitive skills. 

0 

These findings should be placed in the context of both the nature o^f ,the concepts 

and the characteristics of the sample. The concepts were purposely selected to be 

* f * - 

homogeneous. A heterogeneous set of concepts might show less stability. As Well, 
the sample cff college students may be better able to adapt judgments to differing 
'criteria than is the general public. These questions are yet to be explored. Overall, 
putting *«6ide the differences in the distributions resulting from the different pairs, 
the fact that the structures as relative wboles were consistent under the variations 
imposed makes it clear that this procedure is exceptionally robust. Thus, given 
a sufficient sample size and care, with extreme values, it-^ is reasonable to assume - 
that accurate representations of the general public would be obtained. 
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TABLE 2 . 

Intercorrelations of Mean Distances . 
Among Concepts'for "all Treatments^* 



None 
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CC50 • 
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FMIO ' 


rii25 



The n in'each cell is 78.^ ~~ " - 

All correlations are significant p<.0001 
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Children ' s ' Comedy 
Adui't Situation Comedy 
Soap Opera ' 
Family Drama 
Medical Drama 
Crime Drama ? 
Fat Albert , "• ' 
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Thie Walton's, 
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^'Figure 1. Individual plots of each treatment, 
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♦ Figure 2. , Comparison of treatments. Begining at concept number, each point 
represents the judgement of that concept using different' criterion 
jpair. 'The order of * treatments from outer to inner is: FMIOQ, CClOO, 
^ FM50, CC50^ FM25, 0025", FMlO, CC 10" -('•None" treaOicnt not included, . 
see Pig. 3). * * ' 
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APPENDIX A 



Sample Demograpfiics by Treatment 
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